Electronic Journal of Statistics 
Vol. (2006) 
ISSN: 1935-7524 

Bayesian nonparametric estimation of 
the spectral density of a long memory 
Gaussian time series 

Judith Rousseau and Brunero Liseo 

Universite Paris Dauphine, France 
Sapienza Universitd di Roma, Italy 
e-mail: rousseauOceremade .dauphine . f r ; brunero . liseoOuniromal . it 

Abstract: Let X = {Xt, t = 1, 2, . . . } bo a stationary Gaussian random 
process, with mean EXt = fJ, and covariance function 7(t) = E{Xt — 
IJ,){Xt+T — m)- Let /(A) be the corresponding spectral density; a stationary 
Gaussian process is said to be long-range dependent, if the spectral density 
/(A) can be written as the product of a slowly varying function /(A) and 
the quantity \~^'^. In this paper we propose a novel Bayesian nonparamet- 
ric approach to the estimation of the spectral density of X. We prove that, 
under some specific assumptions on the prior distribution, our approach as- 
sures posterior consistency both when /(■) and d are the objects of interest. 
The rate of convergence of the posterior sequence depends in a significant 
way on the structure of the prior; we provide some general results and also 
consider the fractionally exponential (FEXP) family of priors (see below). 
Since it has not a well founded justification in the long memory set-up, 
we avoid using the Whittle approximation to the likelihood function and 
prefer to use the true Gaussian likelihood. 
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1. Introduction 



Let X = {Xt, t = 1,2,...} be a stationary Gaussian random process, with 
mean EXt = /i and covariance function 7(t) = E{Xt — iJ.)(Xt+T — 1^)- Let /(A) 
be the corresponding spectral density, which satisfies the relation 



7(r)= / /(A)e**^dA (r = 0, ±1, ±2, . . . ). 

J — TT 

A stationary Gaussian process is said to be long-range dependent, if there exist 
a positive number C and a value o? (0 < o? < 1/2) such that 

Alternatively, one can define a long memory process as one such that its spectral 
density /(A) can be written as the product of a slowly varying function /(A) 
and the quantity X~^'^ which causes the presence of a pole of /(A) at the origin. 

Interest in long-range dependent time series has increased enormously over 
the last fifteen years; (4) provides a comprehensive introduction and the book 
edited by (10) explores in depth both theoretical aspects and various appli- 
cations of long-range dependence analysis in several different disciplines, from 
telecommunications engineering to economics and finance, from astrophysics 
and geophysics to medical time series and hydrology. 

Pioneering work on long memory process is due to (21), Mandelbrot and 
Wallis (1969) and others. Fully parametric maximum likelihood estimates of d 
were introduced in the Gaussian case by (11) and (<S) and they have recently 
been developed in much greater generality by (10); a regression approach to 
the estimation of the spectral density of long memory time series is provided 
in (12); generalised linear regression estimates were suggested by (■!). However, 
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parametric inference can be highly biased under niis-spccification of the true 
model: this fact has suggested semiparametric approaches: see for instance (25). 
Due to factorization of the spectral density /(A) = A^^"* fW^ ^ semiparametric 
approach to inference seems particularly appealing in this context. One needs to 
estimate d as a measure of long-range dependence while no particular modeling 
assumptions on the structure of the covariance function at short ranges are 
necessary: (19) consider a Bayesian approach for this problem, while (2) provides 
an exhaustive review on the classical approaches. 

Practically all the existing procedures either exploit the regression structure 
of the log-spectral density in a reasonably small neighborhood of the origin 
(25) or use an approximate likelihood function based on the so called Whittle's 
approximation (27), where the original data vector X„ = {Xi,X2, ■ ■ ■ , Xn) gets 
transformed into the periodogram /(A) computed at the Fourier frequencies 
Aj = 27r j/n, j = 1, 2, . . . , n, and the "new" observations ■ ■ ■ , /(A„) are, 

under a short range dependence, approximately independent, each I{Xj)/ f{Xj) 
having an exponential distribution. This is for example the approach taken in 
(7), which develop a Bayesian nonparametric analysis for the spectral density of 
a short memory time series. Unfortunately, the Whittle's approximation fails to 
hold in the presence of long range dependence, at least for the smallest Fourier 
frequencies. 

In this paper we propose a Bayesian nonparametric approach to the estima- 
tion of the spectral density of the stationary Gaussian process: we avoid the use 
of the Whittle approximation and we deal with the true Gaussian likelihood 
function. 

The literature on Bayesian nonparametric inference has increased tremen- 
dously in the last decades, both from a theoretical and a practical point of view. 
Much of this literature has dealt with the independent case, mostly when the 
observations are identically distributed. The theoretical perspective was mainly 
dedicated to either construction of processes used to define the prior distribution 
with finite distance properties of the posterior, in particular when such a prior 
is conjugate, see for instance (15) for a review on this, or to consistency and 
rates of convergence properties of the posterior, see for instance (13) or (26). 

The dependent case has hardly been considered from a theoretical perspec- 
tive apart from (7), who deal with Gaussian weakly dependent data and, in a 
more general setting, (14). In this paper we study the asymptotic properties 
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of the posterior distributions for Gaussian long-memory processes, where the 
unknown parameters are the spectral density and the long-memory parameter 
d. General consistency results are given and a special type of prior, namely the 
FEXP prior as it is based on the FEXP model, is studied. From this, consis- 
tency of Bayesian estimators of both the spectral density and the long memory 
parameter are obtained. To understand better the link between the Bayesian 
and the frequentist approaches we also study the rates of convergence of the 
posterior distributions, first in a general setup and then in the special case of 
FEXP priors. The approach considered here is similar to what is often used in 
the independent and identically distributed case, see for instance (13). In par- 
ticular we need to control prior probability on some neighborhood of the true 
spectral density and to control a sort of entropy of the prior (see Section 3); 
however the techniques are quite different due to the dependence structure of 
the process. 

The gist of the paper is to provide a fully nonparametric Bayesian analysis of 
long range dependence models. In this context there already exist many elegant 
and maybe more general (in the sense of being valid even without the Gaussian 
assumption) classical solutions. However we believe that a Bayesian solution 
would be still important because of the following reasons. 

i) By definition, our scheme allows to include in the analysis some prior 
information which may be available in some applications. 

ii) While classical solutions are, in a way or another, based on some asymp- 
totic arguments, our Bayesian approach relies only on the (finite sample 
size) observed likelihood function (and prior information). 

iii) We are able to provide a valid approximation to the "true" posterior dis- 
tribution of the main parameters of interest in the model, namely the long 
memory parameter d or the global spectral density. 

Also, on a more theoretical perspective, we believe that this paper can be useful 
to clarify the intertwines between Bayesian and frequentist approaches to the 
problem. 

The paper is organized as follows: in the next section we first introduce the 
necessary notation and mathematical objects; then we provide a general theorem 
which states some sufficient condition to ensure consistency of the posterior 
distribution. We also discuss in detail a specific class of priors, the FEXP prior, 
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which takes its name after the fractional exponential model which has been 
introduced by (23) (see also (24) to model the spectral density of a covariance 
stationary long-range dependent process. The FEXP model can be seen as a 
generalization of the exponential model proposed by {'<) and it allows for semi- 
parametric modeling of long range dependence; see also (4) or (17). In Section 
3 we study the rate of convergence of the posterior distribution first in the 
general case and then in the case of FEXP priors. The final section is devoted 
to discussion of related problems. 



2. Consistency results 

We observe a set of n consecutive realizations X„ = {Xi, . . . , Xn) from a Gaus- 
sian stationary process with spectral density /o, where /o(A) — | A|~^''''/o(A). 
Because of the Gaussian assumption, the density of X ^, can be written as 

^^°^^"^= |r„(/o)|V2(2,r)"/2' (2-^) 

where T'„(/o) — ["/{j — fc)]i<j,A;<n is the covariance matrix with a Toeplitz struc- 
ture. The aim is to estimate both /o and do using Bayesian nonparametric 
methods. 

Let ~ {/,/ symmetric on [— tt, tt], / |/| < oo} and ~ {f E f > 0}; 
then JF_|_ denotes the set of spectral densities. We first define three types of 
pseudo-distances on J-^. The KuUback-Leibler divergence for finite n is defined 
as 

KLMo-J) = - / ¥'/o(ZJ[log^/o(Z„)-log^/(Xj]dX„ 

= ^ (T„(/o)r-i(/) - id) - logdet(r„(/o)T-i(/))} 

where id represents the identity matrix of the appropriate order. Letting n — > oo, 
we can define, when it exists, the quantity 



1 r 

A'ioo(/o;./) = - / 



dX. 



/(A) " /(A) 

We also define two symmetrized version of KLn, namely 

hnifo, I) = KLMo; f)+I<Ln{f; /o); rfn(/o, /) = min{A-L„(/o; /), A'i„(/; /o)} 
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and their corresponding limits as n — > cx): 



■TT 



r/o(A) , /(A) 



-2 d\; 



[/(A) /o(A) 



difoJ) 



min{A'Loo(/o; /), KL^{f; fo)}. 



We also consider the L2 distance between the logarithms of the spectral densities, 



This distance has been considered in particular by (22). This is quite a natural 
distance in the sense that it always exists, whereas the L2 distance between / 
and /' need not, at least in the types of models considered in this paper. Let tt 
be a prior probability distribution on the set 



T^{f eTJ{X) = \X\-'^f{\), f eC\--<d<-}, T+ = {f eTJ>0}, 



where is the set of continuous functions on [— tt, tt]. 

Let = {/ G T+;d{f,fo) < e}. Our first goal is to prove the consistency of 
the posterior distribution of fa, that is, we show that, as n — *■ 00, 



where denotes the posterior distribution associated with the prior tt. 

From this, we will be able to deduce the consistency of some Bayes estimators 
of the spectral density / and of the long memory parameter d. Wc first state 
and prove the strong consistency of the posterior distribution under very general 
conditions both on the prior and on the true spectral density. Then, building 
on these results, we will obtain the consistency of a class of Bayes estimates of 
the spectral density, together with the consistency of the Bayes estimates of the 
long memory parameter d. The already introduced FEXP class of prior will be 
then proposed, and its use will be explored in detail. 

2.1. The main result 

In this section we derive the main result about consistency of the posterior distri- 
bution. We also discuss the asymptotic behavior of the posterior point estimates 
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of some parameter of major interest, such as the long memory parameter d and 

the global spectral density. 

Consider the following two subsets of T 

g{d,M,m,L,p)= (2.3) 
{/ e :F+; /(A) = IXr^'fiX), m < /(A) < A/, \f{x) - f{y)\ < L\x - yj"} , 

where -1/2 <d< 1/2, m,M,p> 0; 

T{d, M, L, p) = (2.4) 
{/ e T- /(A) = |Ar2'i/(A), |/(A)| < |/(x) - ~f{y)\ < L\x - j/l"}. 

The boundedncss constraint on / in the definition of g{d, M, m, L, p) is here to 
guarantee the identifiability of d, while the Lipschitz-type condition on /, in both 
definitions, are actually needed to ensure that normalized traces of products of 
Toeplitz matrices, that typically appear in the distances considered previously, 
will converge. We also consider the following set of spectral densities, which is 
of interest in the study of rates of convergence: let 

£*(M,m,L) - {h{-) > 0, < TO < h{-) < M, \h{xyh{y)\ < L\x-y\i\x\A\y\)''} 
and 

C{d,M,m,L) = {/= |A|-2'^/(A),/ e £'^(Af,TO,L)}. 

Note that Q and C are similar, with only a slight modification on the Lipschitz 

condition. The set C has been considered in particular in (2l!). 

We now consider the main result on the consistency of the posterior distribution. 

Let 

Q{t, M, TO, L, p) = yJ-i/2+t<d<i/2~tQ{d, M, TO, L, p) 

and 

C{t, M, TO, L) = U_i/2+t<d<i/2-t^(rf, M, TO, L). 

In the following theorem and in its proof we consider spectral densities in sets 
either in the form Q or in the form C. To simplify the presentation we give 
results for densities in Q only, however the results remain valid for densities in 
£, the only difference being that in the conditions in the form 4|d — do\ < 7 
where 7 = p A po A 1/2 the quantities p,po can be chosen equal to 1 if the 
corresponding spectral densities belong to £. 
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Theorem 2.1. Assume that there exist (to, Mo^mo, Lq) such that we have 

• either fo G G{to, Mq, toq, Lq, po) with < pa < I 

• or /o G £(to,Mo,mo,Lo) = U_i/2+f„<d<i/2-to'C(d, A/o, mo, Lq). 

Let t, TO, M, L be positive reals with t < {p A /Oo)/4- Let n be a prior distribution 
on either Git, M, m, L, p), p > or C{t, to, M, L). If the prior satisfies: 

i) Ve > 0, 7r(Se) > 0, where 

B,^{fe f+it, M, to) : hifo, f) < e, 8(do - d) < 7 - i} . 

iij For ffl/Z £ > 0, small enough, there exists Tn C such that < e^"*" 

and a smallest possible net 7i„ C Q{t, M,m, L, p);d{f ,do) > s/2} (resp. 
C{t, M, TO, L) ) such that when n is large enough, V/ G J-„r\A^, 3fi G 7i„, 
= > / such that 4(d, - d) < p A 1/2. 

27r7_^ /o(a;) 
- //4(d~do) >7-* 



- //4(do-d) >7-* 

Denote by Nn the logarithm of the cardinality of the smallest possible net 
Tin- Then, if 

Nn<nci, with ci > e| loge|~^, 

then 

^"[^elZj^l, /o a.s. (2.5) 
Proof. Sec Appendix B. □ 

The above theorem is important to clarify which conditions on the prior dis- 
tribution TT are really crucial in a long memory setting, where the techniques 
usually adopted in the i..i.d. case, cannot be used and even the adoption of a 
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Whittle approximation is not legitimate in this setting (at least at the lowest 
frequencies). From a practical perspective, however, the hardest part of the pro- 
gram is actually to verify whether a specific type of priors actually meets the 
conditions listed in Theorem 2.1. We will discuss in detail these issues in the 
context of the FEXP prior in §2.3. 



2.2. Consistency of estimates for some quantities of interest 

We now discuss the problem of consistency for the Bayes estimates of the spec- 
tral density. The usual quadratic loss function for the class of functions is 
not the natural one for this problem, since there exist some spectral densities 
in that are not square integrable (i.e. if d > 1/4). A more reasonable loss 
function is the quadratic loss on the logarithm of /, as defined by (2.2), which is 
always integrable, at least in the framework considered in the paper. The Bayes 
estimator of / associated with the loss £ and the prior tt is given by 

/(A) = exp{i?-[log/(A)|Xj} = |A|-2'^'exp{ii;-[log/(A)|Xj, 

where d = £^'^[(i|X^]. Note also that the Bayes estimator of / associated with 
the loss h{., .) and the prior tt is given by 



Also, in many applications of long memory processes, the real parameter of 
interest is just d, the long memory exponent. It is possible to deduce, from 
Theorem 2.1, that the posterior mean of d, that is the Bayes estimator associated 
with the quadratic loss on d, is actually consistent. Let 

J-+(t,M,m) ==U_i/2+t<d<i/2-t{/e •^+,/(A) = |Ar2'^/(A),0 < m<f<M}. 
Corollary 1. Under the assumptions of Theorem 2.1, for all e > 0, as n s- oo, 



{f^\X\-"'f;\d-do\>e}\X„ 



fo a.s 



id d do, fo a.s. 



Proof. The result comes from the fact that, when jd — do| > £i there exists 
a positive constant e' such that for all /, /o G T+{t,M,m), f = |A|^^''/ and 
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/q |A| ^'^"fo, /o) > e'- In fact, assume without loss of generality, that 
d> do, then for all A > AM/m 



HfJo) - (///o + /o//-2)(AdA 



> 



8ttM 



The above quantity is infinite if 2{d — do) > 1, otherwise 



KfJo) > 



47rM(l - 2(d- do)) 



> 



^l-l/(2(d-<io)) 



47rAf(l - 2(d- do))' 
This implies that 

7r[A^,|X] > TT [{/ = IXr^"!; \d-do\ > e}|X„] ^ 0, fo a.s. 

Since d is bounded, a simple application of the Jensen's inequality gives 

{d-dof < E^[{d-dof\Xj ^ 0, fo a.s. 



□ 



It is also possible to derive consistency results for the point estimate of the 
whole spectral density: 

Corollary 2. Under the assumptions of Theorem 2.1, if f2 is as defined in 
(2.6), as n ^ oo, 

hifo,n)^0, foa.s. 

Proof. To simplify the notations we set C to be a generic positive constant. Let 
H{x) — X + x^-^ — 2, then for any a > 



KfoJD 



c 



< c 

= h+h 



Vmh/fW\2Ln] ^E-[f/fo{X)\x, 
E^[Hif/fo{X))\X.J + C I H 



-2]dX 



v/g-[///o(A)|Xj 
Vmfo/fi\)\Xj 



dX 



We have: 



h < Ce + CE"" 



H{f / fo{X))dX\X„ 



\>a 
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C Ce 



Now, consider the test (/>„ defined in the proof of Theorem 2.1 and the same 
type of inequality as those used in the proof of the same theorem: for all f G 

Ef[l -(!)„] < e-"<^|i°s'=l"\ 

Then we choose a small S > such that 

- / / H{f/foiX))E] [1 - dn{f)dX 

J X>a J 

< + C'e^"^|i°g^rV2 1+ f H{f/fo{X))dTr{f)dX 

J \>a 

Let a = exp(— rie| loge|~^/8) then 
We also have 



h = 



H 



VE^[f/fo{X)\X~] 
v/i?-[/o//(A)|X„] 



dX 



< C 



^E^[X-^id-do)\xj ^S-[A-2(<io-d)|x, 



0<A<a ^S-[A-2('io-<i)|X„] ^S-[A-2(d-do)|X„] 



'=dX 



< C A2*A^''j£;^[A-2rf|XjdA + C / A-2'i"A''w'£;^[A2'i|XjdA 

Jo<\<a Jo<X<a 

Let A be the set where d — E'^[d\2£.n\ converges to do! then — 1 and 

V(5 > and n large enough, 



h < c j x'^°-^-'^/^+*dx + c 

Jo<\<a JO<\<a 
< g-ncc|log£pi 



A-'^o-i/^-^dA 



for some c > 0. □ 
Corollary 3. Under the assumptions of Theorem 2.1, as n oo, 

^(/o,r)^0, fo a.s. 

Proof. Note that for all x G K, + e"^ - 2 > x^. Then h{f, fo) > l{f, fo) and 
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This implies, together with the fact that l{f, fo) is bounded when / e Q{t, m, M, p), 
that Ve > 0, 

Kl /o) < E-[lif, /o)|Xj < e + CP^lA^Mn]- 

□ 

Since the conditions stated in Theorem 2.1 arc somewhat non standard, they 
need to be carefully checked for the specific class of priors one is dealing with. 
Here we consider the class of Fractionally Exponential priors (FEXP), and we 
show that these priors actually fulfill the above conditions. 

2.3. The FEXP prior 

Consider the set of the spectral densities with the form 

/(A) = |l-e*^r2''/(A), 

where log /(A) = ^f^o&j cos{jX), for some finite fc G N, and assume that the 
true log spectral density satisfies log/o(A) = J^jLo ^oj cos(jA) (in other words, 
it is equal to its Fourier series expansion), with 

|/o(A)-/o(A')|<i^^^^, E 1^0,1 <oo, 

for all A and A' in [— 7r,7r]. In this section our base model is presented in a 
slightly different way: however it comes to the same thing since |1 — e*'^|/|A| is 
continuous and strictly positive on [— 7r,7r]. 

This class of densities has been considered, from a frequentist perspective, 
in ( ; ). Note that there exists an alternative and equivalent way of writing a 
FEXP spectral density in which the first coefficient of the series expansion is 
explicitly expressed in terms of the variance of the process, that is = 2n e^" . 
We will use both the parameterizations according to notational convenience. 
A prior distribution on / can then be expressed as a prior on the parameters 
{d,K,do,...,6K) in the form p{K)T:{d\K)T:{e\d, K), where 6 = (6*0, 61^), and 
K represents the (random) order of the FEXP model. Usually, d is set inde- 
pendent of for given K and it is also independent of K itself. Let 7r(d) > 
on [—1/2 + 1/2 — t], for some t > 0, arbitrarily small. Let K he & priori 
Poisson distributed and, conditionally on K , in order to obtain a Lipschitz con- 
dition on X^jLi cos (jA) we consider S's such that — where 
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B is large but finite. This implies in particular, that the terms X^jLi l^jl 
uniformly bounded over the supports of ttk- A possible way to formalize it, is 
to assume that, for given K, the quantity Sk = has a finite support 

distribution; then, setting Vj = j\dj\/SK, j ~ one may consider a 

distribution on the set {z e M^; z = (zi, zk), = > 0} for example: 

(Vi, . . . , Vk) ^ Dirichlet(ai, . . . , uk), 
Since the variance of the \9j\^s should be decreasing as j increases, we may 
assume, for example, that, for all j's, aj = + j)^^)- Note that if we fur- 
ther assume that Sk has a Gamma distribution with mean aj and vari- 
ance then we are approximately assuming (modulo the truncation at 
A) that (l^il, . . . , l^fcl) are independent Gamma(l,aj) random variables. Alter- 
native parameterization are also available here; for example one can assume 
that {Vi, ■ ■ ■ , Vfc) follows a logistic normal distribution (1), which allows for a 
more flexible elicitation. Under the above conditions on the prior, the posterior 
distribution is strongly consistent, in terms of the distance d{-,-), the estima- 
tor / as described in the previous section is almost surely consistent and so is 
the estimator d. To prove this, we need to show that the FEXP prior satisfies 
assumptions (i) and (ii). First, we check assumption (i): let ifg be such that 
ISI < %/^/4, then h{fo,foe) < e/8, where 

/o. = |1 - e'^r^"" cxp |e^o, cosjX^ . 

Let = {00,..., Ok,) be such that J2f=o\^aj - Oj\ < j = 1,...,K^ If 

\d-do\<ec, with c < (/^^ |1 - e'^\-^^^dX^ ^ /(Stt), then 

h{fJo)<f r \l-e^^r/'dX + e^^/'^<e 
sir 4 

for e > small enough. Also nK^{{e : \0j - 0aj\ < y/^/{8K,)yj < K,}) > 0, as 
soon as A > J^j l^ojl- Thus assumption (i) of Theorem 2.1 is satisfied. 
Now we verify assumption (ii). Let e > and set 

k 

/M,e(A) = |l-e-*^r2''exp{^0,cos(jA)}, 

j=o 

where the 0j's satisfy the above constraint. Consider 

= {.fk.dM,de [-l/2 + i,l/2-t],/c < fc„}, 
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where fc„ ~ kon/ logn. Since t:{K > fc„) < e , for some r depending on fco, 
we have that 7r{J-^^ > kn) < e~"^ . Now consider spectral densities in the form, 

A: 

/,(A) = (1-eosA)-''' exp{-daog(2)+^0}cosjA}. 

j=o 

Consider 

k 

/(A) = (l-cosA)^''exp{-dfog(2) + ^6ijCosjA}, 

where di — Cie < d < di, 9^ ~ < do + {di — d) fog(2) < Oq — Cge, and 
<coe. Then 

1^ = (1 - cos A)'^'-'' exp{(d, - d) \og{2) + V(0, - 0}) cos jA} 
< 1 

and 

> (l-cosA)-^i'=e-("=+^°)' 

Hence by choosing co,ci,C2 smaU enough, fi — f verifies the three inequahties 
considered in assumption (ii) of Theorem 2.1. The covering number of J-n with 
balls defined by the above inequalities can be bounded by 

exp(7V„) < fc„(Cfc„/e)'^"+2 < e2''^on(-loge-loglog«) 

so that if n is large enough 

N^<ne\\oge\~^ 
and assumption (ii) is satisfied. 



3. Rates of convergence 

In this section we first provide a general theorem relating rates for the posterior 
distribution to conditions on the prior. These conditions are, in essence, similar 
to the conditions obtained in the i.i.d. case; in other words there is a condition 
on the the prior mass of KuUback-Leibler neighbourhoods of the true spectral 
density and an entropy condition on the support of the prior. We then present 
the results in the case of the FEXP prior. 
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3.1. Main result 

We now present the general Theorem on convergence rates for the posterior 
distribution. 

Theorem 3.1. Let {pn)n be a sequence of positive numbers decreasing to zero, 
and Bn a hall belonging to {G{t, M, to, L, p) U C{t, M, m, L)) , defined as 

Bn{5) = {fix) = \x\-^^''-^°^f{x)-KL^{fo;f) < p„/4, 6„(/o, /) < p„, \d-d„\ < 6}, 

for some p G (0, 1]. Let ir be a prior such which satisfies conditions (i) and (ii) 
of Theorem 2.1. Assume that: 

(i) . There exists 5 > Q such that 7r(S„((5)) > exp{—npn/2\. 

(ii) . For all e > small enough, there exists a positive sequence (e„)„ decreasing 
to zero and T,, C D {/, d{f, fo) < e}, such that ^{T^ n {/, d{f , fo) < e}) < 

g-2«p„ _ 

(iii) . Let 

Sn^j = {/ e Tr,; el] < /i„(/o, /) < el{j + 1)}, 

with Jn > j > Jo, with fixed Jq > and Jn = [e^/e^J. VJo < J < Jn, there 
exists a smallest possible net Hn.j C Snj such that \ff G Snj, 3/; > / G Tin.j 
satisfying tv {TMr^T^ih) - id) /n < hMoJ^)/S ,andtr (T„(/, - f)T-\fo)) /n < 
/i„(/o, fi)/^. Denote by Nnj the logarithm of the cardinality of the smallest pos- 
sible net Tin. 

Nn,j < ne'^j"', with a < 1. 
Then, there exist M, C, C" > such that if pn < and n is large enough 

ES [tt (/;/i„(/o,/) > Mel\X)] < max (^e""^-^, . (3.1) 

Proof. Throughout the proof C denotes a generic constant. We have 

/ Vf{XJ/^f„{XJd7r{f) 
If h If f \ l\,r 2 \ Y \ f-''Mo,f)>Msl 

J V/(Z„)Mo(Z„)d7r(/) 
/ ^f{X„)/^fAXjd7r{f) J V'fiKJ/ffoiXjdnif) 

f--e>hr,(hJ)>Mel f:h„{fo,f)>e 



= jr- + Rn,2, 
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for some e > 0. Theorem 2.1 implies that Pq [i?„.2 > e^"*] < for some 
constants C,S > 0. Then wc consider the first term of the right hand side of 
the above equation. Using an argument similar to the one used in the previous 
proof, let 



and 



EH 



D„ 



f--elj<h,,{foJ)<elU + l) 



3>M 



dnif) 



where ipnj = maxj.^-^^ . ipi, and fi is a test function defined as in the previous 
Section, that is tpi = lloi , where 



A = - T-\fo))Xn > tr (id - T„(/o)T-i(/0) + hn{fo, .f^)/^} ■ 

Then, (B.3) implies that 



We also have that 



E, 



(1 - 'Pn^j )- 



n,J 



< PS [Dn < e-"'"V2] + 26""" n {/ : /o) < e}) 
+ 2e"^" / E}' [1 - ^„.,] dnif) 

< 2e-"''" + 2e"''"e-"^^"J'' + Pq" [Dn < e^""" /2] . 
Moreover, using the same calculations as in the proof of theorem 2.1 



< 



where f]„.i = {(Z„, /); X\(7^,7n/) - 7^,7H/o))^„ - logdet[A(/o, /)] < np„/2}. 
Using the exponential bound similar to (B.3), we obtain, if / G Bn, that 

P„"Kl<oxp(-„p..(^^j^Al))<e-/- 

on Bn and Theorem 3.1 is proved. □ 
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The conditions given in Theorem 3.1 are similar in spirit to those considered 
for rates of convergence of the posterior distribution in the i.i.d. case. The first 
one is a condition on the prior mass of KuUback-Leibler neighborhoods of the 
true spectral density, the second one is necessary to allow for sets with infinite 
entropy (some kind of non compactness) and the third one is an entropy con- 
dition. The inequality (3.1) obtained in Theorem 3.1 is non asymptotic, in the 
sense that it is valid for all n. However, the distances considered in Theorem 3.1 
heavily depend on n and, although they express the impact of the differences 
between / and /o on the observations, they are not of great practical use. For 
these reasons, the entropy condition is awkward and it cannot be directly trans- 
formed into some more common entropy conditions. To state a result involving 
distances between spectral densities that might be more useful, we need to con- 
sider some specific class of priors, namely the FEXP priors, as defined in Section 
2.3. For this class we obtain rates of convergence in terms of the L2 distance 
between the logarithm of the spectral densities, l{f, /'). The rates obtained are 
the optimal rates up to a log n term, at least on certain classes of spectral den- 
sities. It is to be noted that the calculations used when working on these classes 
of priors are actually more involved than those used to prove Theorem 3.1. This 
is quite usual when dealing with rates of convergence of posterior distributions, 
however this is emphasized here by the fact that distances involved in Theorem 
4 are strongly dependent on n. The method used in the case of the FEXP prior 
can be extended to other types of priors. 

3.2. The FEXP prior - rates of convergence 

Here we apply Theorem 3.1 to the FEXP priors, which we define through a 
slightly different parameterization. In particular, /(A) = |1 — e''^|^^'*/(A), and 
log /(A) = X^jLo cosjA. Then the prior can be written in terms of a prior on 
{d, K, do, ■■■,0k)- Define now the classes of spectral densities 



S{p,Lo) ^{h> O;log/i e L^[-7r,TT],\ogh{x) - ^ 0, cos ja;, ^ flf (1+jf < Lq}, 



with /3 > 0. Also, assume that there exists a real value (3 > such that /o € 
C*{M, TO, L) n S{P, Lq)- We can then write /o as 
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Note that /? is a smoothness parameter. Classes similar to Lq) arc eon- 
sidered by (22). We now describe the construction of the FEXP prior, so that 
it can be adapted to Lo). Let Sk be a r.v. with density gA{'), which is 
positive in the interval [0,yl], let r/j ~ djj^ and suppose that the prior on 
ivi/^K, ■■■,'r]K / Sk) has positive density on the set 

K+l 

Sk+1 = {x = {Xi, ...,XK+l); ^ Xj = 1} 

. We denote this class as the class of FEXP(/3) priors. Note that if > 1/2 
then there exists a constant M and p < 2/3 — 1 such that for all / e S{f3, Lq) 
associated with the parameters (fc, 9o, Ok) then 

k 

Y,\SA<M, \\og f {x)- log f{y)\<M\x~y\P. 



i=0 



first, 



J2m < ^*2'^02^^ia.|]i|,,i>(i+,).,e,^ 

i^O i^O i^O 



oo 



and, second, since J^jJ'^l^jl '^^ uniformly boimded, 

|log/(x)-log/(y)| < la; -2/1" (2^10,1 I ^10,1/ 



< \x-y\P\^2Y,\9,\jJ2\^,\f 

< M\x-y\ 



p 



Therefore the prior lies in Q {t, m, M, L, p) for some positive constant m, M, L, p. 

We now give the rates of convergence associated with the FEXP(/3) priors, 
when the true spectral density belongs to 5(/3, Lq). 

Theorem 3.2. Assume that there exists 13 > \ s.t. /o G C*{e^° ,e~^° ,L) C] 
S{P,Lq). Let t: be a FEXP{j3) prior and assume that 

i) K Poi{p); 
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ii) the prior on d is strictly positive on [— 1/2 + t, 1/2 — t], with t > 0; 
Hi) Sk has a positive density on (0,^) with A such that > Lq. 



Then there exist C, C" > such that, for n large enough 



and 



{/ G T+ : lU, fo) > C7i-2^/(2/3+i) logn(2/5+3)/(2/3+i)}|x„ < ^ (3.2) 



EH 



KfJo) 



< 2C71-2/J/(2/3+l) J (2/3+3)7(2/3+1) 



(3.3) 



where\ogf{X) = E^ [log/(A)|X, 



Proof. Throughout the proof. C denotes a generic constant. The proof of the 
theorem is divided in two parts; in the first part, we prove that 

C 



EH 



{/ : Kif.h) > n"2^/(2'3+i)logn(2/'+3)/(2/3+i)|x„}] < (3.4) 



while in the second part we prove that 



hnifjo) < Cn-WTT logn^'P l{f,fa) < Cn'— lognWT, 



(3.5) 



for some constant C" > 0, when n is large enough. The latter inequality implies 
that 



EAKf.h)\X^\ < C'n-WTlognwf + 



KfJo)d7T{f\Xj 



A(n,/3) 



2/3 23 + 3 

< 2Cn W+Tlogn^W+i. 



_ 2/3 2/3 + 3 

for large n, where A{n, f3) = {hn{f, fo) > Cn ^jj+t logn^/^+i }. This would imply 
Theorem 3.2. To prove (3.4), we need to show that conditions (i)-(iii) of Theorem 
3.1 are fulfilled. Condition (ii) is obvious because the prior has the same form as 
in Section 2.3 and, because 5(/3, L) C Q{t, m, Af, L', p), with t, m, L', p positive 
constant depending on /3, L. Thus we can choose 



f 



/(A) 



|l_e'^|-2<iexp I ^0j Cos(jA) 



with K < Kn,\d- do\ < S, J2j P'^^] < Lo, leading to 

7T (.f„ n {/, h{f, fo) < e}) < 7t{K > K,,) < e-^" '"s^" 
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for Kn large enough. By choosing A'„ ~ kon^^^^'^^^^ logn^/^^^^^-', we obtain 

Hence, letting p„ = el = ri-2/5/(2/3+i) iogn(2/9+3)/(2/9+i)^ condition (ii) is satis- 
fied. We now show that assumption (i) of Theorem 3.1 is satisfied. Let d < do < 
d + en/ log n^/^ and, for alH = 0, . . . , A'„, 

\0i -eoi\<il + l)-(^+i/2)(log (/ + l))-ie„/logn3/2. 
Since /o G S{(3, Lq), 3to > such that 

^ < LoK-'^ < Celilogn)-^ ^ |0o/| < K-'° (3.6) 



Since 



KLMo-J) < KifoJ) 



= ^ tr (r„(/o - f)T-\f)TMo - .f)T-\fo)) , 

it is enough to prove the assumption under the above conditions for hn{f, /o) < 
Ce^. The difficulty here comes from the strong dependence on n of the distance 
hn- Let 

/o„(A) = ll-e^^r^'^oexp (^00/cosaJ, 6„(A) = 1-exp ( - ^ OiocoslX 

\l=0 / \ 1>K„ + 1 

and gn = fonifon - /); then /o - / = fobn + Un9n and 

nhMoJ) < tr(r„(/o5„)r-i(/)T„(/o5„)T-i(/o)) 

+ tr (r„(/o„<?„)T-i(/)r„(/o„5„)r-i(/o)) . (3.7) 

Both terms of the right hand side of (3.7) are treated similarly, using Lemma 3 
we can bound them by 

tr{Tnifonbn)T-\f)T„ifonbn)T-\fo)) < C (log n\br,\l + 0{n'). 
tr(T„(/o„6„)T-i(/)T„(/o„fe„)r-i(/o)) < C {log n)^ n\ g + 0(12'). 

This imphes that hn{fo,f) < C'e^, when / satisfies the conditions described 
above and 

Bn C j /fc.d.e; k>Kn,d<do<d+ ^" ,0<l<Kn, 
I (logn)'^/^ 

W,-0n,\< 



(log (; + i))iogn3/2 
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The prior probability of the above set is bounded from below by 

Tr{Kn)ni (?7i, ...,?7x„) : \vi - Voi\ < G- — pnlogri.-^/^ 

\ (log/) log ri''/^/ 

where denotes the uniform measure on the set Jy/^n); X); ^7? — 

We finally obtain that 

7r(S„(5)) > e-'^-^"'°s« > e-np™/2 

by choosing fco small enough, and condition (i) of Theorem 3.2 is satisfied by the 
FEXP(/3) prior. We now verify condition (iii) of Theorem 3.2. Let jo < J < Jn^ 
where jo is some positive constant, and consider / g S'nj , as defined in Theorem 
3.1, where /(A) = je,k = |1 - e'^l-^-i exp{X;ti fi*; cos (/A)}. Consider 

fc k 

MX) = |l-e'^r2'^"exp{^0rcos(/A)};/(A) = ll-e'^r^'^ exp{^ 0, cos (/A)}, 

1=1 1=1 

then, if c > Co E;>o(^ + l)"'^"^/^ log (/ + iy \ co > and >d>d'' - celj, 
one obtains 



1 < ^(A) < (1 - cos A)-2«'^e5«"^' 



and 



tr (T-i(/)T„(/„ - /)) < 15c6^j tr {T-\f)Tn{fu)) 

< Ccelj 

< CchnifoJu)- 

Choosing c small enough one obtains that tr (T~^ (/)T„(/tj — /)) < n/i„(/o, /„)/8. 
Similarly 

tr {T-\fo)TMu - ./)) < 4ce2j tr {T-\fo)TMu)) 

< cCKifoJu)/S- 

Since we are in the set {f\h{f^,f) < e}, for some e > fixed but as small as 
we need, there exists e', e" > such that 

K 

\d-do\<e', Y.{0,-eiof+ J2 ^io<£"- 

1=1 1>K+1 
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Let K < K„ — A'on^^^^^^^-'(logn)^^, the number of fu defined as above in the 
set Sn,j is bounded by 

and 

Nnj = logiV„.j < Cjuel 

where Cj is decreasing in j. Hence by choosing jo large enough condition (iii) is 
verified by the FEXP(/3) prior. This achieves the proof of (3.4) and we obtain 
a rate of convergence, in terms of the distance hn{-, •)• We now prove (3.5) to 
obtain a rate of convergence in terms of the distance Consider / such that 

hnifo, /) = ^ tr {T-\fo)TM - fo)T-\f)TM - /o)) < 
Equation (C.2) of Lemma 3 imphes that 

^ tr {TMo')Tn{f - fo)Tn{f-')TM - h)) < Ce^ 

leading to 

- tr (T„(go)r„(/ - /o)T„(g)T„(/ - /o)) < Cel (3.8) 
n 

where go = {1 ~ cos A)''", 5 = (1 — cos A)''. 

We now prove that tr [Tn{go{J — fo))Tn{g{.f — fo))) < Ce^: we use the same 
representation as in the treatment of 7(6) in Appendix C. For the sake of sim- 
plicity we consider the case d > do 

A = -tr(T„(.go(/-/o))T„(.g(/-/o))) 
n 

- - tr (TMTM - fo)T„ ig)TM - fo)) 
n 

= - / (/-/o)(A2).9o(A2)(,/-,/o)(A4)g(A4)f^^^- 

[-7r,7r]3 

X A„(Ai - A2)A„(A2 - A4)A„(A4 - Ai)dA 
+ - / (,/-/o)(A2).9o(Ai)(/-/o)(A4),g(A4) 

[-71-, Tt]* 

X A„(Ai - A2)A„(A2 - A3)A„(A3 - A4)A„(A4 - Ai)dA 
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< / |A2r^('^-*)|Airi+^L„(Ai-A2)*i„(A2-Ai)dA 



+ 

n 



n 

[— -7r,7r]- 

C f |Ai| 



2d I 



I. lo^-, |i„A -^"('^l ^ '^2)in(A2 — A3)L„(A3 — A4) LniXi — Xl)dX 



-7r,-7r 



^ C log nn 



2S 



n 

Clognn^ f |Aii2'' 



< g(logn) 



n 

[—IT, TV 

2 



/I Ai I I 
1^ |2(j^ p_g -^rt(Al — A2)i„(A2 — A3)i„(A3 — Xi)d\ 



if (5 > 4((i — do). We have used inequality (C.6) together with incquahty (C.5). 
This implies, together with (3.8) that 

1 



- tr (r„(go(/ - h))Tn{g{f - /o))) < Ce 



1 



n 



To finally obtain (3.5), we use equation (C.3) in Lemma 3 which implies that 
A„ ^ tr(T„(go(/-/o))r„(g(/-/o)))-tr(T„(.gog(/-/o)')) 
< Cn-'+' + log l\ei\ ( [ gogif- fofiX)dx] . 

Moreover 



-r/(2/3-l) 



1=1 



by choosing r = (2/3 - l)/2/3, A„/n is of order „-(4/3'+i)/(2/3(2/3+i)) ^j^j^j^ jg 
negligible compared to 77,-2/3/(2^+2) ^^.^ if /3 > 1/2 

.9o.9(/o - !fdX < el 

which achieves the proof. □ 

4. Discussion 

In this paper we have considered the theoretical properties of our Bayesian 
procedure. A related and important problem, which deserves the same attention. 
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is its practical implementation. Due to the length of the present paper, we 
discuss these issues elsewhere; see for example (2U); here we only sketch the 
main features of the proposed algorithm. 

From a computational perspective, the practical implementation of a non- 
parametric Bayesian analysis based on a FEXP prior and a Gaussian likeli- 
hood, is plagued by two difficulties: i) the number of parameters to estimate 
varies with K the number of terms in the FEXP expansion; ii) the likelihood 
function is quite expensive to evaluate, due to the Toeplitz structure of the 
covariance matrix. 

After trying several approaches we finally recommend the use of the Z3-kernel 
Population MonteCarlo algorithm, presented and discussed in (9), and which 
can be easily adapted to the varying dimension set-up. For the evaluation of the 
inverse and of the determinant of the Toeplitz covariance matrix, we have used 
the algorithms proposed in (G). 



Appendix A: Lemmas 1 and 2 



We state two technical lemmas, which are extensions of (18) on uniform con- 
vergence of traces of Toeplitz matrices, and which are repeatedly used in the 
paper. 

Lemma 1. Let t > 0, M > and M a positive function on ]0,7r[, let p be a 
positive integer, and 

T{d, M, M) = J / e F, Vm > 0, sup < M{u) \ , 



|A|>t. 



dX 



we have: 



sup 

p(di+d2)<l/2-t 
/,6J^(di,M,M) 
g,G.F(d2,M,M) 



" \i=l J i=l 



and let L > and p G (0, 1] 

^tr (J{TM^)Tn{9^) \ - (27r)2f-i | (A).g. (A)dA 



sup 

p(di+cl2)<l/2-t 



O(A.l) 



0(A.2) 
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This lemma is an obvious adaptation from (1 s), and the only non obvious part 
is the change from the condition of continuous differentiability in that paper 
to the Lipschitz condition of order p, considered equation A. 2. This different 
assumption affects only equation (30) of (f8), with replaced by 77^, which 
does not change 

Lemma 2. 



sup 

2p(di-d2)<P2Al/2- 
/iG:f(di,M,L,pi) 
gi€G{d2,m,M ,L,p2) 



sup 

2p(di-d2)<P2Al/2- 
fie:Fidi,M,M) 
gi€G{d2,m,M ,L,p2) 

and 



sup 

2p(di-d2)<l/2-i 

fie:Fidi,M,M) 
gi&C{d2,m,M,L) 

Proof. In this second lemma, the uniformity result is a consequence of the first 
lemma, as in (L'>); The only difference is in the proof of Lemma 5.2. of (8), i.e. 
in the study of terms in the form 

|id-r„(5)'/'T„((4^2^ri)T„(g)i/2|. 



Following Dahlhaus's (S) proof, we obtain an upper bound of 

g(Ai) 

5(A2) 

which is different from (8). If .g G Q{d2,m, M, L, P2), the Lipschitz condition in 
p implies that 

Calculations using Ln as in (n) imply that 

\I - T^ilY'^T,, {{AiT^f)-^) T„(/)i/2|2 = 0{n^-^nogn) + 0(n*), V<5 > 0. 



convergence results. 

-trfnT„(/,)r„(g.)-'Vi- r n^dA ^0, 
' \T.i ) 2ir J_, Ji9,(A) 

;.r(nT,.(/.)r„ter')-^£ng^.A|^0^ 

.(nT,.(/.,r„,..r.)-^£nf^.A|^o^ 



9{y) 
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li g £ C*{M, m^L) as defined in Section 3.2, then 



f{y) 



< K 



\x-y 



1-35 



(k|A|2/|) 



1-5 



< K\x - y 



1-35 



and (n) Lemma 5.2 is proved, leading to a constraint in the form 4p((ii — (^2) < 1 
(corresponding to p = 1). 

Then, using again Dahlhaus' (1989) calculations, we obtain that 



and finally that 



itr (f[A,-f[B)i 

\i=i i=i / 



k=l 

which goes to when 2p{d2 — di) < p A 1/2. 



p 

^0(r^"^/^"^(P"''^('*""''l)"2('i2-dl)„l/2-p^| 

fc=i 
p 

k=l 



2(p-fe+l)(d2-di)-(pAl/2)^ 



□ 



Appendix B: Proof of Theorem 2.1 

Before giving the proof of Theorem 2.1, we give a few notations that are used 
throughout the paper: Let /, /i be spectral densities: 

. A(/i,/) = r„(/)-iT„(/i) 

. S(/i,/) -T„(/i)i/2[T„(/)-i -r„(/i)-i]T„(/i)V2 
. 6„(/i, /) = tr (id - T„(/i)T„(/)-i)2) /n 
. fe(A,/) = (2^rV:.(/i//-l)'(^)rf^- 
Proof of Theorem 2. 1 

The proof follows the same ideas as in (13). The main difficulty here is to 
transform constraints on quantities such as ft.n(/, /o) or KLn(f, fa) in terms of 
distances between /, /o independent on n, uniformly over /. 

We can write 

Then the idea is to bound from below the denominator using condition (i) of 
the Theorem and to bound from above the numerator using a discretization of 
based on the net 7i„ defined in (ii) of the Theorem and on tests. 
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Let e > S > 0: one has 



= P1+P2 (B.l) 

Also, let 

Bnic) = {/ e g(t, Af, m, L, p) : nKL„{fo, /) < nc}, c > 0. 
Using Lemma 2, when n is large enough, 

Bn{6/2) D{fe g{t, M, m, L, p); /i(/o; /) < -,8{do - d) < p A 1/2 - t} = 65/4 
so that assumption (i) implies that, for n large enough, 
7r(S„(<5/2)) > n{Bs/i) > e-^''^ 12. 

Define 

a. ^ {(/, X) : -X*[T„(/)-i - r„(/o)-i]X + log (clet(A(/o, /))) > -n5}. 
We then have 

V. < pAI ^^4/)<e--/^^) 

7r(S„)\ 



< PS\^{B.nr\VLn)< 



2 

7r(S„)^ 



2 



Moreover, 



Po"[f}^] = Po" (X^[T„(/)-i - T„(/o)-i]X„ - log (det(A(/o, /))) > nS) 
= Pr[2;*P(/o,/)2/-tr(P(/o,/)) 
> 71(5 + log (det(A(/o, /))) - tr (P(/o, /))], 

where y ~ iV„(0, id). When / e 6„, n5 + log (clet(A(/o, /))) - tr (P(/o, /)) > 
n6/2, 
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SO that 



< Pr[y* B{foJ)y -triBifo J)) >n6/2] 
4E[{y'B{foJ)y-tT (i?(/o,/)))1 



< 



< 



tr {B{fo,f)^)C 



Therefore, for all f G Br, 



and 



Pom < 



Pi < Ci/n^ 



M'C 



where Ci is a positive constant. 

We now consider the second term of (B.l), namely: 



(B.2) 



< e-"('-2^-)+p2, 

take 2(5 < r and consider p2- Consider the following tests : let fi £ Tin, 

(j), = ll^,(j,-i(^^^_^-i(^^-|^^>^^^. 

Recah that 7 = po A p A 1/2 - t (or p A 1/2 - po A 1/2 - t, 1/2 - t 
depending on whether the spectral densities belong to Q or C). We now prove 
that < e-"'^|i°s^l"' and E'jll - (f),] < e-"=l i°g'^l"' for / close to 

1. If A\do - d,\ < 7, set = tr (id - T„(/o)T-i(/0) /n + Kifo, /,), then for 
all 1/4 > s > 0, 



<exp{-snp,}E^ 



o'^X'JT-\fo)-T-\f,))X^_ 



= exp {-snp,} exp{-- logdet[id + 2sS(/o, 

< exp {~snp, - s tr (B(/o, /,)) + tr (((id + .srS(/o, .U)r'B{fo, f, 

< exp {-snp, -str (B(/o, /,)) + 4^^ tr {B{fo, f^f)} , 
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where r G (0, 1) and the latter inequaUty is due to 

id + srBifo, h) - id(l - 2st) + 2sTA{fo, h) > ^id, if s < 1/4. 

Replacing pi by its above expression and choosing s to optimize the latter 
expression, we obtain: 

Eo m < max I cxp{-n ^g^ ^^ },exp{-n } 1 . (B.3) 

Uniformly on the support of tt, 

lim 6„(/o, .U) = ^ I (1 - k/hf{x)dx ^ 6(/o, /,), 

lim hn{fa,fi) = /i(/o,/,;)- 
Therefore, for any a > 0, if n is large enough 

E, [0.] < max (^exp{-n^^^^^^-^^},exp{-n }j , 

choosing a < since /i € we obtain 

E, [^,:] < max {-"^^(^(^-^y^^}> -P {-n^^} j . (B.4) 

Lemma 4 implies that if e > is small enough, there exists a constant Ci 
such that 

i?o"M <exp(-nCie|log(£)|-i). 

Moreover, if / is in the support of 11 and satisfies / < fi, and 4|(io — d| < 7, 
using the same kind of calculations as in the case of i?o I'^d ^^'^ ^^'^^ 
that 

id - 2</2(/)(r-i(/0 - r-i(/o))TV2 > id + 2sBif, /o), 

we obtain if < s < 1/4, 

E] [1 - 0.] < e"^''- exp{-s tr /o)) + tr /o)^)} 

< cxp{-ns/i„(/o, /,) + 4s4 tr /o)') + s tr - /, /o))}. 

Using Lemma 2 if (27r)-i / (/, - f)f^'^{x)dx < h{fQ, f,)/A, when n is large 
enough (uniformly in /) tr {A{fi - /, /o)) < nhn{fo, fi)/'^ and 

[1 - 0,] < max e^"32E(7-7^ ^ e""^^ j . (B.5) 
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Again Lemma 4 implies that if e > is small enough, there exists a 
constant Ci > such that 

2. If 4(d, - do) > 7- Set = tr (id - TMo)T-Hn)) /n + Xi„(/o; the 
upper bound of i?o [t/'i] is computed similarly to (B.4) so that 



< max exp{-n— ————}, exp{-n 1 

V obn[Jo,Ji) 4 

Now, using the same calculations as in Dahlhaus (1989, p. 1754), there 
exists a constant C > 1 such that 

T^T / f f \ ^ bnjfo, fi) 
J<^n[JO,Ji) > ^ 

SO that, for large n (independently of fi), 



KLMo,.h) > 



2C 

We finally obtain that there exists a constant c > such that 

£^0 ^ <cxp{-ncfo(/o,/,)}. (B.6) 

Since 6(/o, /;) is uniformly bounded from below on the set 

{/ e g{t, m, M, L,p);d>do + l/4(p A po A 1/2) - t/A} 

(or £{t, m, Af , L)), if e is small enough 

£^oM <cxp{-ne}. 

Consider / < fi, such that A{di — d) < /? A 1/2 — Similarly to before, let 
h e (0,1): 

Ef [1 - (t>,] < e(l-'')"'''/2-^l°gdot[id-(l-ft)Ty^(/)(T-i(/.)-T-i(/o))Ti/=] 
< g(l-h)np,/2-i logdct[id-(l-/i)B(/,/o)] 

^ g(l-h>ip,/2-logdGt[^(/,/o)]/2-ilogdct[id(l-/i)-/iT-i/2(/)T„(/o)T-i/2(/)]^ 

then using the same kind of expansions as in (B.5), we obtain 

i?7[l-0z] <det[A(/„/)]i/2 

nKLnUoJif \ ( nKLn{fo,fi) 



X max < exp ■ — , , , cxp 

^ 32tr(i3(/o,/)2)/n ' ' 
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Since logdct[A(/, /O] = - logdct[id + T„(/, - using a Taylor 

expansion of log dot around id , wc obtain that for n large enough 

- logdet[A(/, /O] < / (/, - f)/f{x)dx + a 



27r . 

where a can be chosen as small as necessary. Also 



Since 



if 



32tr(i?(/o,/)2)/n - b{foJ) + a 



b{fo, /) < 26(/o, fi) + -{ I f^ir' - f-^f{x)dx 



TT 



1 

2^ 



if^~f)/f{x)dx<cb{foJ^)/4, 



there exists a constant ci > such that when n is large enough 

E] [l~<l>^] < exp(-nci6(/o,/0) <e-"^ 

for e small enough. 
3. If 4(do - d,) > 7. Sctp, - logdct[T„(/,)T„(/o)-i]/n, then if < /i< 1 

< g-nKL„(/,Jo)+/i"tr(B(/.,/o)2) <- g-ne 

where the last inequality can be obtained by following the same lines as 
for (B.6). 

Moreover, for all / < fi, satisfying 4{di — d) < p A 1/2 — if 

^ f {h-f)lf{x)dx < c6(/„ /o)/4, i- f^\f-nf{x)dx < bin, /o), 

where the constant c is defined such that, for n large enough, for all / in 
Q{t, TO, M, L, p) (resp. C{t, m, M, L)) such that 4,[dQ—d) > 7, KLn{fi, fo) > 
c6(/i, /o) (sec the calculations presented in the case 4((i — do) > 7), 

[1 - < exp{-2sn(A'i„(/„ fo) - tr - /, /)) /n) + As^nb„{f, fo)} 
for £ small enough. 
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Then, in each case, we have, for large n (mdcpendcntly of fi), 
for all e < eo, 

Let 0^") = maxi 0^; then, using Markov inequality, 

< g-«e|logerV2^ 

We finally obtain that for some (5 > 0, if n is large enough 

for some positive constant Cq, so that 7r[yl^|X„] P§° a.s. 
Appendix C: Lemma 3 

Lemma 3. Let fj, j e {1, 2} be such that fj{X) = \\\^^''-^ fj{X) , where dj < 1/2 
and fj G S{L,f3), for some constant L > and consider b a bounded function 
on [— 7r,7r]. Assume that /in(/i,/2) < e where e > 0. Then V(5 > 0, there exists 
eo > such that if e < eo , there exists C > such that 

i tr (T„(/i)-ir„(/i6)r„(/2)-iT„(/i6)) < C{\ognf[\b\l + \b\l,n'-^ + n-'/^+\ 
n 

(C.l) 

itr(r„(/r^)r„(/i-/2)T„(/2-i)r„(/i-/2)) < ChMiJ2). (c.2) 

Lei = (1 — cos A)''^ and fj ~ gj^fj^ where fi e S{L, P)r]C and /2 e /?), 
written in the form log /2(A) = X^zLo '^'-'^ e-^)/ then 

i tr (r„(5i(/i - f2))TMh ~ /2))) - tr {T^{gm{h - 12?)) 

< Cn-'+' + n-HognY,m\( f gmifi - f2)H^)dx] , (C.3) 
for any 5 > 0. 
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Proof. Throughout the proof C denotes a generic constant. We first prove (C.l). 
To do so, we obtain an upper bound on another quantity, namely 

lib) = itr(T„(/fi)T„(/i6)T„(/2-i)T„(/ife)). (C.4) 

First note that b can be replaced by |&| so that we can assume that it is pos- 
itive. Let A„(A) = ^"^iGxp(—iXj) and L„ be the 27r-periodic function de- 
fined by i„(A) n if |A| < 1/n and L„(A) = \\\-^ if 1/n < |A| < tt. Then 
I An (A) I < CL„(A) and we can express traces of products of Toeplitz matrices 
in the following way. Let the symbol dX denote the quantity dXidX2dX3dX4; 



7(6) = 



C f /i(Ai)6(Ai)/i(A3)fe(A3) 



Jl-^M^ ./o(A2)./o(A4) 

A„(Ai - A2)A„(A2 - A3)A„(A3 - A4)A„(A4 - Xi)dX 
C f /i(Ai)/i(A3) ,,,, ,2 , ,,,, , 



I f a U a ^ (6(Ai)2 + 5(Ai)fe(A3)-6(Aif) 

n fo[X2)fo[Xi) 

xA„(Ai - A2)A„(A2 - A3)A„(A3 - A4)A„(A4 - Ai)dA 
^ tr (r„(/i52)T„(/f i)r„(/i)T„(/2-i)) 



n 



" J[-7r,7r]4 /l(A2)/2(A4) 

xA„(Ai - A2)A„(A2 - A3)A„(A3 - A4)A„(A4 - Ai)dA 

On the set 6(Ai) > 6(A3), < 6(Ai)-6(A3) < 5(Ai) and on the set 5(A3) > 6(Ai), 
< 5(A3) — b{Xi) < ^(Aa), therefore the second term of the r.h.s. of the above 
inequality is bounded by (in absolute value) 

^(h\ < ^ f /i(Ai)/i(A3)6(Ai)% 

7(oj < — / , w /, N -t-n(Ai - A2j^„(A2 - A3) 



xL„(A3 - A4)L„(A4 - Ai)dA 
xL„(A3 - A4)Ln(A4 - Xi)dX 



Note that 



i„(Ai - A2)i„(A2 - A3)dA2 < ClognL„(Ai - A3), (C.5) 
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therefore 



lib) < 



-C 



'IX l-2d 



1 



1 



^2| / Vl^4 

xi„(Ai - A2)i„(A2 - A3)i„(A3 - A4)£„(A4 - \i)d\ 



+2C 



6(Ai)2|Ai|-2('^i-rf2) 



|-2di 



-7r,7rJ* \IA2l |A4| 

xi„(Ai - A2)£„(A2 - A3)L„(A3 - A4)L„(A4 - \i)d\ 



|-2di 



|A4l-2rf2 



- 2 



Since 



|Ai|-2'i. 



lAsI-^'^. 



- 1 



<C^i^-^, for, = {1,2}, 



(C.6) 



using Dahlhaus' (1989) calculations as in his proof of Lemma 5.2, we obtain 
that, if di - d2 < (5/4, 



5(A,)^|A,|-(^-''^) 



- 1 



|A: 



\-2d.2 



IA4I 



-2ds 



- 1 



X L„(Ai - A2)i„(A2 - A3)L„(A3 - A4)i,i(A4 - Ai)dA 

i?i(Ai — A2)L„(A2 — A3)''L„(A3 — A4)i„(A4 — Ai)'' 



[— 7r,7r]* 
,2<5|> |2 



|Ai|i-*/2|A4|i 



-5 



< Cn''\b\U\ogny 



as long as \di — ^2! < S/2. By considering /i„(/, /o) < e with e > small enough, 
we can impose that \di — ^2! < and we finally obtain that 

7(&) < C\b\l{lognf + C\b\ln''"\lognf. (C.7) 

We now prove that, for large n and V5 > 0, 

-tr(T„(/i)-ir„(A6)r„(/2)-iT„(/i&)) <citr(r„(/,'i)r„(Afe)r„(/2-i)T„(/i&))+Cn 

Since /,;(A) < C|A|^2d, ^ ^^.(;s^)^ i = 1,2. This implies that T-i(/,;) t C-^T-\gi) 
so that we can replace by T^^{gi) in the above term. Then 

5n = tr {TMib)T-\gi)Tn{fib)T-\g2)) 

= tr {TMib)Tn{g^'/{in'))TMibn)Tn{92'/{'^7T')))) 

+ tr (T„(/ife)r-i(,9i)r„(/ife)r-i/2(g2)^2y-i/2(^^2)^ 

+ tr (T„(/i&)T„(.gi)-i/2i?iT„(gi)-i/2T„(/i6)T„(g2'V(47r2)) 



-1+5 
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where R, = T„ig,y^^T„{g^' /i47:^))T„ig,y/^ - id, i = 1,2. Using (C.7) we 
obtain that 

tr(r„(/ifo)r„(5fi)r„(/i6)r„(.g2-i)) < C{lognfn\bn\l + Oi\b\oon^)=n-f. 
Moreover 

< \R2\\T~''\92)TMib)T-\9i)Tn{hh)T~''^{g2)\ 

< <5y2|i?2|||T-V2(^^)T„(/,5)i/2||||T„(/ife)V2r-V2(5^)|| 

Lemmas 5.2 and 5.3 in Dahlhaus (1989) lead to: V(5 > 

tr (T„(A6)T-i(5i)T„(/i6)T-i/2(g2)i?2T-i/2(g2))| < Cn^+^f'^"-'^) |6„U<5y' < CSl/^ 
Similarly, 

tr (T„(/i6)T„(.gi)-i/2i?ir„(.gi)-^/'r„(/i&)r„(grV(47r2))) 
< \Ri\5l!^[\Ri\ + l]||r„(gi)-V2T„(/i6)i/2||||T„(/i&)V2T„(g^)-i/2|| 

Since ||r„(5i)i/2r„(g-V(47r2))i/2|| < l + \\TM"^Tr.{9i' I -^^W < 

for all 5 > and using Lemma 5.3 of Dahlhaus (1989) 

tr (T„(/i6)T„(.gi)-V2^ir„(.gi)-i/2r„(/i6)r„(g2-V(47r2))) | < Cn^'Si/'. 
Finally we obtain for all (5 > 0, when n is large enough 



and (C.l) is proved. We now prove (C.2). since fj > m\X\ 



'2di 



gj where 

T-Kfo) -< T-\gj), i.e. T-\g,) - T-\f,) is positive semidefinite, 



and 



Kih. /2) - ^ tr (T„(/i - f2)T-\h)T^{h - f2)T-\h)) 
> ^ tr (r„(/i - f2)T-\h)Tn{h - f2)T-\gi)) 



> 



i-tr (t„(/i - /2)T-i(/2)T„(A - h)T-'/'{gi)RiT-'^\gi)) 



(C.8) 
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2n 



tr T„(/i - f2)T-\g2)Tn{h - /2)T„ 



(9^ 
V47r2 



2n(167r4 



■ tr (r„(/i - /2)T„(g2-i)T„(/i - /a)^^ (.gfi)) 



+ 7^ tr fr„(/i - f2)T-\h)TMi ~ f2)T~^'\gi)RiT-^'\gi)] (C.9) 



2n 



+ 1^^^^ tr (r„(/i - f2)T-'^H92)R2T-'^'{92)Tn{fi - h)Tn (.9r')) 



where = iA~ Tn''^{gj)Tn{gJ^ /{ATT'^))Tn''^{gj). We first bound the first term 
of the r.h.s. of (C.9). Let (5 > and e < eo such that |(i — (io| < 5 (Corollary 1 
implies that there exists such a value eo). Then using Lemmas 5.2 and 5.3 of (8) 



tr 



(t„(/i - f2)T~\f2)T„{h - f2)T~"^{gi)RiT-"\gi) 

< 2|i?i||r-V2(g,)r„(/,_/2)r-i/2(/,)|||T„(|/i-/2|)V2r-V2(j^)ll 
X \\Tn{\fi-f2\y/'T-'/'{gi)\\ 

< Cn''\T-'/' {g,)T,, {h-h)T-''^ {h)\- 



Since gi < Cfi , 

\T-'/'{gi)TMi-f2)T-'/'{.f2)\' = tr {T-\g,)TMi-f2)T-\f2)TMi-f2)) 

< ctT (r-i(/i)T„(/i-/2)T-i(/2)r„(/i-/2)) 

= Cnhn{fi,f2), 

and 



tr (TMi-f2)T-\f2)TMi-f2)T;;'/'igi)RiT-'/\9i)) | < Cn''-'^'-hMiJ2). 
We now bound the second term of the r.h.s. of (C.9). 

i tr [TMi~f2)T-'/'{g2)R2T-'/\g2)TMi~f2)Tni9^') 

< ^|i?2||T-i/^g2)r„(/i-/2)r„(.9i)-^/'||T„(5i)i/2r„(5r^)T„(|/i~/2|)T-V2(j^)| 

< ^"'^"^"^^''^^^ l|rn(gl)^/^y.(.gr^)rn(|/l-/2|)r-^/^(/2)|| 



n 

x||T„(5i)-i/^r„(|/i-/2|)i/^||||T„(|/i-/2|)i/X"i/2(/2)|| 
< Cn^'-^/^hMiJ2), 
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Since ||T„(/i)i/2r„(/fi)r„(A)i/2|| < ||id|| + |r„(/i)i/2T„(/fi)r„(A)i/2-id| < 



Therefore 
C 



n 



tr {TMi~f2)Tn{92')Tn{fi~f2)Tn{9Y')) < C K{h, /2)(1 + n-^'^+^'), 



and, using the fact that C gj > fj, for j ~ 1,2 this proves (C.2). The proof of 
(C.3) is similar: 

^ = tr (r„(5i(/i - h))Tn[92{h - h))) - tr (r„(gig2(/i - hf)) 

= C y ffi(/i-/2)(Ai)[ff2(/i-/2)(A2)-5i(/i-/2)(Ai)]A„(Ai-A2)...A„(A4-Ai)dA 



= C y .gi(/i-/2)(Ai)(/i-/2)(A2)[g2(A2)-52(Ai)]A„(Ai-A2)A„(A2-Ai)dA 

[— 7r,7r]^ 

- C y <?i52(/i-/2)(Ai)[/i(A2)-/i(Ai)]A„(Ai-A2)A„(A2-Ai)dA 

[-7i-,7r]2 

+ C [ gi52(/i-/2)(Ai)[/2(A2)-/2(Ai)]A„(Ai-A2)A„(A2-Ai)dA. 



The first 2 terms of the right hand side are of order 0{n^^ logn). We now study 
the last term, here the problem is due to the fact that /2 does not necessarily 
belong to C. We have: 

/ [-.,.]^3i52(/i - /2)(Ai)[/2(A2) - /2(Ai)]A„(Ai - A2)A„(A2 - Ai)dA 

gig2{fi - /2)(Ai)/2(A2)[g2-i(A2) - .g2-'(Ai)]A„(Ai - A2)A„(A2 - Ai)dA 



- /2)(Ai)[/2(A2) - /2(Ai)]A„(Ai - A2)A„(A2 - Ai)dA. 



The first term of the above inequality is of order 0{n^^ logn) because 52 belongs 
to C. Since 



/(A) =exp (^JicosilX)^ , 
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one gets 

/ = f 5i(/i-/2)(Ai)[/2(A2)-/2(Ai)]A„(Ai-A2)A„(A2-Ai)dA 



< C / gi|/i-/2|(Ai) 



^ 0l{cOS {j\2) - COS (jAi)) 



i=0 



L„(Ai - A2)i„(A2 - Ai)dA 



< C\ogn{^^\ei\l^ J g,\h - hmdX 



I 



1/2 



< C\ognY,\Bi\l 



1=0 



5i52(/i-/2)'(A)dA 



\[-7r,7r] 



where the latter inequahty holds because J gi/g2{X)dX can be proved to be 
bounded by an application of an application of Holder inequality. □ 

Appendix D: Relations between b(/o,/) and h{fQ, f) 

Lemma 4. Let m,AI,L > and p < 1. There exists t > and C > such 
that for any /, /o € g{t, m, M, L, p) U C{t, m, M, L), if h{f, fo) < t, 

b{f, fo) < h{f, f o)\ log h{f,fo)\. 

We need to bound b{f, fo) in terms of h{fo, f) when |d — do] is small. Assume 
that /o = |a;|~^''°/o and / = \x\~'^'^f with d > do (otherwise the bound is 



straightforward) we have 



fS 



dx. 



HfJo)^ 



Let ^ > be large enough; using the fact that m < f, fo < M we obtain, if 
A>2, that 

bifjo) < AhifoJ)+ j ^ fif/fo~lfdx 



< AhifoJ) + 



f/fo>A 

r 2aP 



f/fo>A 
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Let A > M/m then if f/fo > A, |a;|-2(d-do) > Am/M, so that 

KLfo) < AhifoJ)+C J Ixl-'^^-^oUx 

Now assume that /i(/o, f) < t where r > is fixed and small. Consider i > 
small enough so that 

M/o,/)> j 

2.-2(d-do)>i-l 

where > means that the inequality is up to a multiplicative constant whose 
value does not depend on / and /o (but it does depend on M and m). It implies 
that 

_2_,-,«™-..,<cM/./o). 

so that, if t^^ = logl//i(/, /o), 

log(logl//i(/, /o))— -1— > logl/(p/j(/, /o)), p > fixed . 
2{d-do) 

Hence, if h{f, /o) is small enough, 

^ , ^ 21og(logl/fe(/,/o)) 

2(d - doj < ] 77777 — 7^ • 

logl//i(/o,/) 

Now using the fact that there exists C > such that 

^l-2(<i-do) 



b{f, /o) < Ahifo, /) + CA2-i/(2(d-d„)) ^ f)>C'- 



l-2{d-do) 

and considering yl = log /o) we finally obtain 

b{f, /o) < log 1/Hf, /o)/i(/o, /) + C'hif, /o) log /o)). 

Hence, there exists r > (depending only on m,M and C > 0) such that if 
Kf, fo) < T, b{f, /o) < h{f, fo)\\ogh{f, fo)\. 
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